# Transformer-XL Architecture

XLNet is a model pre-trained on English language using generalized permutation language modeling objectives and Transformer-XL architecture, achieving SOTA results on multiple language tasks.

XLNet is an unsupervised language representation learning method based on a generalized permutation language modeling objective, using Transformer-XL as the backbone model, excelling in long-context tasks.